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SOME EMPIRICAL TESTS IN VOCATIONAL 
SELECTION 


1. The Problem. 


The aim of this investigation is to test out in practice — 
the method of empirical vocational tests and to discover, if 
possible, significant tests for specific commercial functions. — : 
; The type of work selected for correlation with the tests 

is typewriting. Data in stenography and grammar, obtained : . 
with one of the groups of subjects, have been utilized and in- ay 
cluded in the investigation. 
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2. History. 


A. Tests for typists. 


Lahy (32)* reports an experiment made with eleven typ- 
ists as subjects; six women who had had from two to four 
years practice, and five men who had had from two to eight 
years practice, in typing. Measures of the following func- 
tions or processes were obtained; muscular sensibility of the 
two hands, speed of auditory reactions, memory for numbers 
and sentences, attention, and several higher processes such 
as imagination, abstraction, judgment and reasoning. In 
comparing the three good and the three mediocre women sub- 
jects it was found that the former posessed a finer threshold 
of tactile and muscular discrimination, a tendency of the 
hands to become equally strong, a better memory, fewer er- 
rors of attention, but showed longer auditory reaction times. 
In comparing the men subjects the same distinctions were 
found. This is an interesting and valuable contribution in 
that it stimulates work along these lines and blazes the trail, 
but it has not been worked out extensively enough to be prac- 
tically applicable. 

Lough (35) reports graphical curves showing the rela- 
tion between the time required to perform a letter substitu- 
tion test, “habit formation test,” measured after repeated 
trials with the test, and ability in stenography and typewrit- 
ing for a group of high school students. The curves show 
a good degree of correspondence between performances in the 
_ test and judgments of ability in stenography and typewriting. 
To quote Loughy “It is evident that if we give these pupils 
a test in habit formation first, we can determine what pupils 
are capable of doing good work in typewriting, and we can 
save some of the pupils a great amount of wear and tear, by 
simply allowing them to take some other work in place of 
typewriting. They are not adapted to the motor responses 
which typewriting calls for. By means of this experiment it 
would be possible to select those who are likely to succeed 
in typewriting and to give a vocation to those who would not 
succeed.” If this test could accomplish all that Lough claims 
for it it would indeed be a boon to the human race but un- 
fortunately, the necessary scientific data concerning the ex- 
periment are not given. There is only one test used and the 


*Refers to Bibliography. yp. 94. 
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investigator who would build up a system of vocational guid- 
ance or selection for such a type of work on the ability to per- 
form any one such test, when the original data are inconclu- 
sive, would be incautious indeed. However, a good deal of 
credit is due Lough for his pioneer effort. 

Link (33) gives a series of tests to detect ability in type- 
writing and a series of tests to detect ability in stenography. 
The tests are, for typing—letter substitution, Trabue com- 
pletion, context reading, spelling and typing; for stenography 
—context reading, mixed letters and numbers, substitution, 
Trabue completion, spelling, grammar, dictation and typing. 
tank states? Ws Ss relevant tests were given to two senior 
classes of over three hundred girls and boys in a commercial 
high school, to twenty-six pupils of two business schools, to 
a group of twenty-two office typists, to another group of nine- 
teen stenographers, to over four hundred candidates for posi- 
tions as typists and stenographers..... the tests selected 
on the basis of these experiments are those which showed the 
highest and most consistent agreement with the abilities 
of those examined.” These are broad statements, made in the ~ 
absence of any norms, methods used, correlations, or any re- 
liable data whatsoever, which should be presented if ‘‘consist- 
ent agreement” is to be proved. 


B. Primitive methods in vocational selection. 
In primitive magic there is the naive idea that by simply — 
expressing a wish forcibly enough the wish will be material- 
ized ; thus if the parents of an Indian boy wished their son to be 
tall, they would toss him in a blanket. In medieval clair- 
voyance there was and is still a belief that there are signs 
which reveal an individual’s character. The sources of these 
signs are principally to be found in the zodiac, palm of the 
hand, tea leaves, cards, dreams, crystal gazing, etc. In the 
nineteenth century there was a spot located in the brain for 
the function of speech and later all parts of the body were 
found to be localized in the brain. Phrenology was a distortion 
of this fact. Signs disconnected with the individual’s body 
are now given up and internal and personal external signs 
take their place. In phrenology there was no comparison of 
skulls, but individuals were taken at random who had some 
peculiarity in the formation of their skulls; for instance, if 


*p. 422-3 and chapter 8. 
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an individual had a bump anywhere on his head and was 
also known to possess some peculiarity such as stinginess, 
then that bump was labelled stinginess. When it became 
known that there were no scientific criteria for phrenology, 
the skull was discarded and the face was taken up, and the 
pseudo-science of physiognomy was born. Physiognomy main- 
tains that there is something in the features which is signif- 
icant of inner hidden traits. This is true in so far as the 
characteristic is an expression of a quality of character. But 
in vocational psychology we are mainly interested in young 
people whose characteristics have not been well developed or 
well stamped upon their faces. 


C. Traditional methods in vocational selection. 


In the selection of employees the employer, for the most 
part, has been content to form a ready estimate of a candi- — 
date’s ability by looking him over, or else has accepted him on | 
the authority of someone who has recommended him; or 
simpler still he tries the candidate at the task and, sink or 
swim fashion, he promotes him or discharges him according as 
he succeeds or fails in the work assigned. None of these 
methods of measuring a man’s value will bear criticism. The 
last is always costly and time consuming; the second is un- 
certain, depending on the degree of confidence that one has 
in the recommender, his ability to evaluate and his motives; 
as to the first there may be some men who are geniuses at pick- 
ing men of worth at sight; they seem to have an unerring in- 
sight which prompts them to their choice, yet even in such 
cases, it appears that one does not know the signs of value 
intuitively but the ability to discriminate has been acquired 
only through long and costly experience. 


D. Modern methods in vocational selection. 
a. Vocational miniature. 


In the vocational miniature, the work is reproduced on a 
small scale apparatus which duplicates the actual situation 
which the worker faces while engaged at his task. Typical of 
this method is an experiment of McComas of Princeton, in 
which he attempted to detect good and bad telephone operat- 
ors. He constructed an actual switchboard on a small scale. 
The operator, to be tested, made connections at the board 
which were timed on a kymograph in an adjoining room. This 
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species of “reaction and co-ordination” time showed the in- 
terval between the appearance of a light over a call connection 
and the moment that the operator “plugged in,” and also be- 
tween the moment that a number was called and the appropri- 
ate connection made. Fifty-one trials were made by each of 
the nine operators tested and the average time of each subject 
‘was compared with the average ranking of the subjects by the 
two supervisors. The test detected the two best and two out 
of the three poorest operators. The correlation of these rank- 
ings with the rankings of the supervisors was .6250 with a 
probable error of .14. This method has the advantage of con- 
creteness and apparent relevance but there is little actual as- 
surance that the ideas and feelings aroused correspond with 
the processes aroused in the actual work. 
b. Vocational sample. 

The method of vocational sampling is closely related to 
the method of miniatures. It simply consists of taking an 
actual piece of the work to be performed and sampling the 
candidate’s ability by the degree of his success in the trial. 
Thus, in testing applicants for jobs as filing clerks, they may 
be given a number of cards to be arranged in order according 
to date, number, alphabetical position, address, etc. In cer- 
tain cases specimens of work have been devised or taken into 
the psychological laboratory and the worker watched more 
carefully and measured more exactly. Unfortunately this is 
.a method which cannot be applied to very many kinds of work, 
for care must be taken to make the tests representative of all 
the work. At its best this method detects only the presence of 
ability and is not a means of gauging potential ability. 

c. Vocational analogy. 

This method consists of an attempt to create, in the lab- 
oratory, a situation which arouses and exercises the same 
mental functions which the actual job arouses and exercises. 
The material is new, but the attitude and endeavor of the 
worker are intended to be the same. Munsterberg stated 
(87) that, by this method, sea captains could be selected who 
could be relied upon to avoid accidents. The test material 
consisted of twenty-four cards, the size of playing cards, on 
which are printed four rows of capital letters namely, A E O 
U in irregular repetition and interspersed with a few other 
letters. The person to be tested had to distribute these twen- 
ty-four cards as quickly as possible into four piles according 
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to the numerical preponderance of the four capital letters.. 
The time taken to sort the cards and the number of errors 
made is taken as the index of ability to safely guide a ship’s 
course. The author of this test admits that there was no 
correlation scientifically determined between the test and the 
actual ability to guide a ship’s course with safety. The insu- 
perable difficulty with this method is that there is no way of 
ascertaining that the same mental functions which are called 
into action on the job are really being exercised and tested 
in the experiment. This is a highly dangerous method and 
has done much to put vocational psychology into disrepute 
in the eyes of men of practical business affairs. 
d. trade tests. 

The method of trade tests is that of putting specific prob- 
lems, which are germane to the work to be performed on the 
job, to the applicant for a job. The answers to the questions 
give an index of his experience and ability on the job. This 
idea has been made use of in the past in the examinations 
of candidates for positions by some concerns and chiefly by 
some civil service commissions. (Appendix 3 for sample . 
trade test.) 

A good trade test should differentiate between various 
grades of skill, should produce uniform results in various sec- 
tions of the country and in the hands of individuals of widely 
different characteristics and should consume the least amount 
of time and energy consistent with satisfactory results. 
While there are all degrees of trade ability among the mem- 
bers of any trade, the method of the trade tests has classified 
all of the members of any trade into a few groups, usually 
four, such as novice, apprentice, journeyman, and expert. The 
novice is a man who has no trade ability whatever, or at least 
none that could not be paralleled by practically any intelligent 
man. An apprentice has acquired some of the elements of the 
trade but is not sufficiently skilled to be entrusted with any 
important task. The journeyman is qualified to perform al- 
_ most any work done by members of the trade. An expert can 
perform quickly and with superior skill any work done by men 
in the trade. It is essential that the trade test differentiate 
between the journeyman and the apprentice, and the appren- 
tice and the novice. Trade tests devised to make this classi- 
fication are of three kinds: oral, picture, and performance. 
The oral tests are most generally used because they are of low 


SOME EMPIRICAL TESTS IN VOCATIONAL SELECTION 11 


cost and may be applied to a large number of men in a compar- 
atively short time without much equipment. 

The following stages are characteristic of the usual meth- 
od followed in obtaining a trade test. 

1. Investigation and collection of trade data. An investi- 
gation is made into the conditions of the trade to determine 
the following facts. (a) The feasibility of a test in the trade, 
(in one instance it was found that the trade of gunsmith was 
not a recognized trade, though there were gun repairers). 
(b) The elements which require and permit of testing to de- 
termine whether men can be graded in the trade according to 
degrees of skill, (in some trades it has been found that the 
trade required simply the performance of a single set of oper- 
ations and there were no gradations among the members of 
the trade). (c) The. kinds of tests that can be used, (some 
trades such as truck driving are mainly matters of skill and 
for them performance tests are better than oral or written 
tests.) Other trades such as interior wiring and power plant 
operation are mainly matters of knowledge. For these trades. 
oral and picture tests are better. After having discovered 
by inquiry that the trade is a recognized trade and can be 
tested, all the information necessary is collected in the field 
from all available sources, such as experts of the trade, trade 
union officials, literature of the trade, trade school author- 
ities, employers and the like. 

2. Compiling the questions. As a result of the informa- 
tion collected, a number of questions, usually forty to sixty are 
compiled, each of which calls for an answer which shows 
knowledge of the trade. The experience of the formulators 
of trade tests has shown that a good question meets the fol- 
lowing requirements. (a) It must be in the language of the 
trade. (b) It must be a unit, complete in itself and requiring 
no explanation. (c) It is not a chance question that could be 
answered by a good guess. (d) It must be as short as possi- 
ble and must be capable of being answered by a very short 
answer. (e) It must not be ambiguous. 

3. Preliminary sampling. After the large number of 
questions originally formulated has been sifted down by the 
application of the requirements listed in the preceding para- 
graph the questions are used in a preliminary sampling on a 
number of tradesmen whose answers indicate the merit of the 
different questions and their grades of difficulty. In this 
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sampling tradesmen from different shops or plants are tried, 
in order to guard against specialized methods or modes of ex- 
pression confined to a single locality. 

4, Revision and formulation. The preliminary sampling 
affords a means of checking on the following points. (a) Is 
the test applicable to trade conditions? (b) Does the test rep- 
resent good trade practice? (c) In what way can parts be 
profitably modified, supplemented or eliminated? (d) Does 
the test represent the whole range of the trade from the novice 
to the expert? (e) Is it a representative sampling of the whole 
range of trade processes? In the light of the answers to these 
questions the test is revised and then formulated. — 

5. Final sampling. Final sampling is made by testing 
a number of men, usually twenty, who are known to be typical 
members of each group (novice, apprentice, journeyman, ex- 
pert.) Among the novices tested are usually some highly in- © 
telligent and mature men of good general knowledge but no 
trade ability. Statistical treatment of the results and of the 
answers to each question enables the determination to be made 
of a relative value of each individual question and the selection 
that makes a proper balance. 

6. Evaluation. If a trade test is good, a known expert 
when tested, is able to answer all, or nearly all, the questions 
correctly ; a journeyman is able to answer the majority; an 
apprentice a smaller part; and a novice practically none. This 
does not mean that each question should be answered correct- 
ly by all the experts, a majority of the journeymen, some ap- 
prentices and no novices. There are a few questions which 
show this result. Other types of questions, however, are more 
common. Some show a distinct line of cleavage between the 
novice and the apprentice. Novices fail, but apprentices, 
journeymen and experts alike answer correctly. There are 
likewise questions which are answered correctly by nearly 
all the journeymen and experts, but only a few apprentices and 
also questions that only an expert can answer. Each type of 
question has its value in a good test. The main requirement 
is that the tendency of the curve of distribution of the scores 
should be upward. A question that is answered correctly by 
more journeymen than experts or more apprentices than 
journeymen is undesirable and is at once discarded. A proper 
balance is made of the others. 

7. Calibration. As each question is allowed a number of 


SOME EMPIRICAL TESTS IN VOCATIONAL SELECTION 18 


points, usually four, it becomes necessary to determine how 
many points should indicate an expert, how many a journey- 
man, etc. This is accomplished by noting how many points 
were scored by the known experts and the known journeymen 
when they were tested. Ordinarily the expert scores higher 
than the journeyman and the journeyman higher than the 
apprentice. It frequently happens that a few journeymen 
score as high as the lowest of the experts and a few appren- 
tices as high as the lowest of the journeymen. There are, con- 
sequently, certain overlappings between the classes. In cal- 
ibrating, the object is to draw the dividing line between class- 
es so that the overlapping shall be as small as possible. When 
these dividing lines, or critical scores have been established 
the test is ready to be applied. 

Picture tests are made in practically the same way as the 
oral tests. The peculiar characteristic of picture tests is 
that the questions making up the tests relate to illustrations 
of trade tools and appliances. 

Performance tests are fundamentally the same as the 
method of the vocational sample discussed earlier. 

Certain important difficulties of the method of the trade 
tests may be mentioned. One is due to the fact that trade 
terms are very fluent, for instance the word “plug” probably 
means a score of different things in many trades, and prob- 
ably no single one of these twenty “plugs” is universally 
known by that name. Localisms exist in various parts of the 
country such as “come along” in the linesman’s trade in New 
York City. Foreigners, who compose no insignificant part 
of the total number of tradesmen, have a different nomencla- 
ture and one which is not usually a literal translation of the 
English. There is a difference in trade procedure in different 
parts of the country, carpenters trained in New York, Georgia 
and California will answer the question “What is done 
first in building a house” in very different ways and each 
in a way which is right according to his training. People may 
be coached, with ease, upon items of specific information—an 
individual with the barest trace of literacy can be taught to 
answer the following written questions: “What is your 
name?” “In what country were you born?” “In: what state in 
the Union is your domicile?” Coaching is quite as easy in other 
specific information tests. Good trade tests determine simply 
the presence of trade experience and the degree of skill actu- 
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ally possessed. They are not prophetic tests which indicate 
probable aptitudes for trades which the person tested has 
never followed. 

e. Miscellaneous empirical tests. 

Finally, there are cases in whch tests having vocational 
significance have been sought by purely haphazard and em- 
pirical ways. These may be designated as empirical vocation- 
al tests. This method consists of selecting at random a num- 
ber of tests and obtaining measures of large groups of work- 
ers in any vocation and finding out if there are any tests which 
have a high positive correlation with the actual work. This 
is a strictly empirical method which is not based upon any 
apriori postulates as are most of the other methods. In the 
homely phrase of Carlyle, these tests seem to be “significant 
of much.” 


SOME EMPIRICAL TESTS IN VOCATIONAL SELECTION 15 


3. Procedure. 


A. Materials employed. 

Nine tests were used, selected at random, from the group 
of tests which Woodworth and Wells (53) have standardized 
for the American Psychological Association. In groups 2 
and 3 it would have been unwise to have attempted to use more 
than this number since it took approximately thirty minutes 
to test a subject. Under commercial conditions this was a 
great amount of time for the concern to give, and also for the 
subjects to give since they were losing their bonus for the 
time spent on the tests. The tests used are—opposites, verb- 
object, agent-action, action-agent, color naming, mixed rela- 
tions, hard directions, number checking and form substitu- 
tion.* Of those tests in which more than one form is given by — 
Woodworth and Wells the form which was used in this experi- 
ment is as follows. 

*Considerable time was spent by many subjects on the stimulus. words 
“love” in the opposites test, on “kiss” in the verb-object test, and on 
“lungs” in the agent-action test. It was evident, that for some subjects, 
it was more difficult to respond to these words than to others of the list. 
If the tests were to be given over again these words should be taken out 


ne others, of the same difficulty as the rest of the list, substituted for 
em. 


B. Individuals serving as subjects. 


| - _ Verb- Agent- Action- ‘ . 
Opposites object ei Sl agent Mixed relations 
north read baby gallops Eye-see Ear- 
sour tear fire bites Monday-Tuesday April- 
out throw dog boils Do-did See- 
weak paint laborer sleeps Bird-sings Dogs- 
good mail pencil floats Hour-minute Minute- 
after light army growls Straw-hat Leather- 
above sail heart sails Cloud-rain Sun- 
sick spin pin roars Hammer-tool Dictionary- 
slow lock gun scratches Uncle-aunt Brother- 
large wash eyes stings Dog-puppy Cat- 
rich bake bird shoots Little-less Much- 
dark spill wind melts Wash-face Sweep- 
front kiss lungs swims House-room Book- 
love polish bell explodes Sky-blue Grass- 
tall sweep musician aches Swim-water Fly- 
open fill parrot glows Once-one Twice- 
summer sharpen clock news Cat-fur Bird- 
new write ax cuts Pan-tin Table- 
come chew broom flies Buy-sell Come- 
male drive mosquito burns Oyster-shell Banana- 


a. Group 1. 
Seventy-seven young men and women, who were studying 
typewriting, stenography, and grammar in the Extension De- 
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partment of Columbia University, served as subjects in this. 


first group. These people were divided among four different 
sections, three evening classes and one day class. In all of 
the evening classes it was impossible to obtain any adequate 
measure of the subjects’ abilities, for they either dropped out 
in short time or failed to take the prescribed tests and exami- 
nations. The results worked up for the day class of forty- 
five students, forty-three female and two male are the only 
ones used in this investigation. The subjects made an in- 
tensive study of typewriting and stenography devoting two. 
_ hours in class to each subject each day for five days a week. 
Some outside study was also done. The course lasted from. 
the first part of October, 1915, until the middle of May, 1916. 
In group 1 the performances in the tests were correlated with: 
abilities in typewriting, stenography and grammar. 


b. Groups 2 and 38. 

One hundred and eighteen typists, all female, in a enea 
retail commercial concern in New -York City were tested. 
Of these ten left the employ of the concern before any ad- 
equate measure of their abilities could be determined. The 
remainder were divided into two groups. Group 2 consisted. 
of thirty-eight typists who had been working in the same 
division for at least ten months. Group 3 consisted of sixty- 
five typists who had been working in this same division for 
at least one month and a half and for not longer than six 
months. All of the subjects in both groups had been doing 
the same kind of typing. The records of five typists who had 
been working between six months and ten months were dis- 
carded. This division was made in order to allow, as well 
as possible, for the effect of practice in the particular kind 
of typing which the subjects in these two groups were per- 
forming, as it would have been unscientific to put all of the 
subjects in one group.* In groups 2 and 8 the performances. 
in the test were correlated with ability in typewriting only. 


c. Age, education and experience of groups 1, 2 and 3. 
The figures in the following table are in per cent of the 
number of individuals in each group. 
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AGE 
| Group 1. Group 2. Group 3 
16-19 years old. 46 .68 91 
0-24 82 o2 .08 
25-29 # .06 .00 01 
- 30-34 “ ll .00 .00 
35-38 “4: .04 .00 .00 


*See section 3, C, c, 2, (description of the work of groups 2 and 3). 


EDUCATION 
Grammar school education .02 61 AT 
1-3 years high school .02 36 AT 
High School graduates 51 .03 .05 
1-3 years college 22 .00 .00 
College graduates 22 .00 .00 
EXPERIENCE 
Never worked previously 85 .00 31 
Worked at typing 1-5 months .00 .00 10 
Worked at typing 10-12 months .00 .66 .26 
Worked at typing more than 1 year .00 34 23 
Worked at other work than typing 15 23 20 


In the age table the mode for all groups is the group 16- 
19 years old. In the education table the mode for group 1 
is the full high school education, for group 2 it is the grammar 
school education, and for group 8 it is either the grammar 
school education or the 1-3 years high school class. 


C. Methods of obtaining measures of abilities. 


a. Stenography. 

In February a mid-year examination was given in stenog- 
‘raphy which was graded by the A, B, C, D, F,.method; A, 
B, C, being the order of satisfactory grades, D a condition and 
F failed. In each of these grades the mark was further 
qualified so that the A group, for instance, came to consist 
of A plus, A, and A minus: The individuals were then ranked 
in an order of merit in each of the three sub-divisions of 
each grade, for instance if there were three subjects in the 
A plus group they were ranked, according to their abilities, 
1, 2, and 8. From these last rankings an order of merit 
series, from one to forty-five, was secured. 


b. Grammar. 

In February a mid-year examination was given in gram- ‘ 
mar, spelling, punctuation, paragraphing, letter writing, etc. 
This was a thorough three hour examination and the results 
were probably a good index to the subjects’ abilities. The 
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grades in this examination were returned in per cent terms 
which were readily transformed into an order of merit series. 


c. Typewriting. 
More objective and reliable grades were obtained in type- 
writing than in either stenography and grammar, the sub- 
jects being measured in actual quantity and quality of output. 


I. Group 1. 

Each month a typewriting test was given by the instruc- 
tor, in which the subjects copied a given piece of material 
on the typewriter, accomplishing as much as they could in 
ten minutes. The total number of words written in ten 
minutes, less five words for each error made, divided by ten 
gave the net number of words written per minute. This is 
an arbitrary method of scoring adopted as the best relation- 
ship between speed and accuracy by the International Type- 
writing Committee and has been incorporated into the inter- 
national rules. The following table gives the number of 
subjects taking the examinations, the average net number of 
words written per minute, standard deviations, and ranges. 


No. of Av. no. Ss. D. in Range in 
subjects words per min. words per min. words per min. 

October 42 16.4 6.2 6. -37. 
November 40 22.5 7.1 9. -51. 
December 40 27.4 6.5 15.6-51.2 
February 40 29.7 (eI 15.0-48.1 
March 29 , $4.1 5.0 23.7-44.5 
April 27 38.9 4.8 30.4-53.9 


II. Description of the work of groups 2 and 38. 

The commercial concern, in which the subjects of groups 
2 and 3 worked, was composed, in part, of a number of stores 
which retailed merchandise and which received all of their 
orders by mail. The customers’ orders, in the form received 
by the concern, were sent to the division in which these sub- 
jects worked, to have a sheet typed for each store in the 
concern from which the customer might have ordered a com- 
modity. Thus an order would require anywhere from one to 
_ forty-six sheets written for it, as the customer might order 
articles, in one letter, from one to forty-six of the stores. 
A single order contained an order for a commodity from but 
one store and required but a single sheet to be written for 
it. Each sheet had typed on it the customer’s name and ad- 
dress and specifications of the commodity ordered, such as 
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amount, size, shape, style, pattern, etc. This was a very 
particular kind of typing since there were practically no 
long phrases or sentences written and many figures, symbols 
and abbreviations were used. Any hierarchy of higher type- 
writing habits which the subjects might possess appeared to 
be of little advantage since it could not be made use of directly. 
All of the subjects in these two groups were working ex- 
clusively on the single orders. 

It was ascertained, from the total number of words writ- 
ten and the total number of strokes made in several samples 
of three hundred sheets each that the amount of work done 
in a day could be expressed in terms of the number of sheets 
written in the course of a day’s work. Thus the number of 
typed sheets per day furnished the measure of the amount 
of work turned out by the subjects. A group of “checkers” 
read each sheet typed by the subjects, compared them with 
the customer’s original order and sent back to the subjects 
every sheet on which an error was made; and the subjects 
corrected the error in the course of the day’s work, every error 
being thus penalized. A system was put into effect in which 
a girl circulated among the subjects and kept each subject 
supplied with orders, never permitting any subject to run out 
of orders. By this means no subject was able to select orders 
and thus receive easy ones. 

A weekly bonus on the number of sheets written was in 
operation in which all of the subjects of group 2 shared al- 
most every week and in which an average of approximately 
forty per cent of the subjects of group 3 shared. There was 
much evidence that each girl was urging herself to her best 
efforts. 


III. Group 2. 

Records of the average number of sheets written per day 
were kept for ten months, from these there were selected the 
records of what appeared to be the best successive five weeks, 
the selection being based upon; 1. Attendance records; no 
one subject being absent more than once during this period, 
in which case she was credited, for the day absent, with the 
average number of daily sheets that she had written for the 
rest of the week in which the absence occurred; 2. Season of 
the year; the season being selected when the business was 
heaviest and when the subjects were being stimulated, by 
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their supervisor, to turn out as many sheets as possible. What 
appeared to be the best week’s work was selected in like 
manner, the best week being a part of the five weeks which 
were selected. The best day’s work was simply that day, 
within the five weeks’ period, in which each individual 
subject did her best work. The five weeks’ measures 
and the best week’s measures were for the same calendar 
weeks for all subjects, but the best day’s measures were 
not the same for all subjects. Thus three measures of 
ability were obtained: 1, daily average for five weeks; 
2, daily average for one week; 3, best day’s performance. 
The number of subjects, thirty-eight, was constant for 
each measure. These three cross sections of the subjects’ 
abilities for different lengths of time were selected simply to 
ascertain whether the tests would correlate to the same degree 
with each. The following table gives the average number of 
sheets typed, standard deviations, and range of the average 
number of sheets typed. | 


No. of Ss. D. of Range of 
sheets typed sheets typed sheets typed 
Daily average for five weeks 450.3 64.8 360.2-662.0 
Daily average for best week 509.5 68.0 402.5-735.2 
Best day’s performance 612.0 68.2 531. -812. 


IV. Group 38. 

Not all of the subjects of this group were working for 
the concern at the same time. Some subjects entered the 
employ of the concern two or three months after others and 
some of these latter had separated from the concern before 
the former had entered. The constant flux of the coming 
and going, of the arrival and departure of the individuals in 
- this group, where the labor turnover was over three hundred 
per cent per year, made it impossible to take any one series of 
weeks’ output, which weeks would be the same for all, as 
measures of abilities, as was done in group 2. A study of 
the records showed that many of the subjects began doing 
very good work about the fifth week after entering the con- 
cern. Consequently, the average number of sheets written 
per day during the fifth and sixth weeks of each individual’s 
term of employment with the concern was taken as the 
measure of abilities. These weeks, then, were not the same 


calendar weeks for all individuals. The best day’s work was — 


also selected as a measure of abilities, the best day being that 
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particular day in which each subject did her best work. The 
following table gives the average number of sheets typed, 
standard deviations, and ranges. The number of subjects, 
sixty-five, was constant for both measures. 


No. of S.D. Range 
sheets typed 
Daily average for 5th & 6th weeks 311.8 62.3 198.4-420.3 
Best day’s performance 361.9 65.3 258. -500. 


D. Some remarks on the procedure. 

The problem of securing groups in which the output of the 
workers is accurately comparable. In typing it is difficult to 
obtain a large group of typists in which all of the individuals of 
the group are performing the same kind of work which can be 
accurately measured, quantitatively and qualitatively. The 
difficulty lies in the fact that the group chosen at random 
is usually a heterogeneous one; that is, one individual does 
specific work which differs from the specific work of another 
individual, although both are performing the same generic 
act—typing. This difficulty has not entered into this investi- 
gation since all of the girls were a part of a homogeneous 
group and all were doing exactly the same kind of typing. 
The measures obtained in this investigation were not subject 
to the indeterminable error of a supervisor but were ob- 
jectively obtained so that it was possible to state that one girl 
was so many points better or poorer than another girl. | 

Testing at various periods of the development of practice. 
In group 1 all of the subjects were tested when they had 
practically no practice or knowledge of typing. In group 2 
all of the subjects were tested after they had acquired con- 
siderable practice and skill in typing. In group 3 the subjects 
were a mixed group being made up of individuals possessing 
_ varying degrees of practice and ability in typewriting. 

Care taken to secure accurate measures of typing ability. . 
More than one measure of typing ability was used in each 
group. In group 1 six measures, in group 2 three measures, 
and in group 3 two measures of ability were used. In group 
2 approximately 9100 records of daily output, and in group 3 
approximately 3000 records of daily output were tabulated 
and studied in order to obtain what appeared to be the best 
measures of typing ability. : 

Wide mental differences of the groups of subjects. The 
wide mental differences between group 1 and groups 2 and 3 
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in educational training indicate that two widely different 
classes in these respects were tested. It appears that the 
highest and lowest mental grades of girls in the typing pro- 
fession were included in this experiment. 


FE. Administering the tests. 


a. General procedure. 

The subjects in group 1 were tested within two months 
after they had started their course. The subjects of groups 
2 and 3 were tested after they had acquired considerable speed 
in typing. All of the tests in groups 2 and 3 and all 
of the tests but the number checking and the form substi- 
tution in group 1 were given individually. The number 
checking and the form substitution tests in group 1 were 
given as class tests. The method of conducting the ex- 
periment was simple. The subject was seated comfortably 
at a table, on which the blanks were presented one at a time, 
after it had been ascertained that the subjects thoroughly 
understood the instructions. In the number checking, form 
substitution and hard directions tests replies were filled in in 
pencil. In the remainder of the tests the replies were spoken. 
In the tests where the replies were spoken the experimenter, 
who sat next to the subject, had a copy of each test and if 
an inaccurate reply was made the subject was required to 
correct it before giving a reply to the next stimulus word, 
the subjects being told before hand that they would have to 
do this. In the number checking test each group was checked 
which contained any combination of the digits eight and nine. 

b. Instructions to the subjects. 

In the instructions to the subjects an effort was made to 
combine the proper comprehension of the experiment by the 
subject with an ideal uniformity of instruction. In the in- 
dividual tests instruction was given by description, illustra- 
tion, and execution. The subject was first clearly told the 
meaning of the test, then the experimenter performed a 
small sample of it, and finally the subject herself performed 
a small sample of the work. These samples were presented 
in typewritten form and the same samples used with all 
subjects. Six illustrations of each test were used with all 
subjects. In the class tests the subjects were simply told 
what they were to do and the experimenter performed a 
small sample of the work on the blackboard. 
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An attempt was made to impress upon the subjects the 
fact that speed was the main object of the tests. They were 
given a short talk on the applications of psychological tests 
and were told that their mental reactions were going to be 
timed and that their individual results were going to be 
compared with the results of the rest of the class. They 
were urged to go as fast as possible but were warned that 
they would either have to correct their mistakes as they went 
along or be penalized later for their mistakes (according 
to whether a class or individual test). It seemed that all the 
subjects tried to make all the speed possible. 


ce. Records. 


In all the tests in groups 2 and 3 and in all the tests but 


the two class tests in group 1 the time was taken by a split 
second watch reading to fifths of a second. The watch was 
started when the experimenter judged that the subjects’ eyes 
met the first figure or word on the test sheet, which was un- 
covered by the experimenter. In the class tests instructions 
were given that as soon as anyone had finished she was to 
raise her hand. When the first hand was raised the experi- 
menter called upon the class to stop, having instructed them 
beforehand that he would do this. Thus the time for all was 
the same and the results were worked up for the amount 
accomplished. : 
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4. Results and how obtained. 
A. Scores in the tests. 


The following table gives the scores made in the tests for 


_ groups 1, 2 and 38. 


Group 1 Group 2. Group 3 
. aes PR eS es) a ae 
Opposites 28.56 8.26 39.00 12.25 36 .48 48.37 16.12 .52 .95 
Verb-object 31.438 7.91 38.59 10.26 .28 .29 37.23 10.89 .18 .31 


Agent-action 31.92 10.09 40.61 16.21 .27 .60 39.81 14.28 .25 .41 
Action-agent 33.06 6.75 46.33 11.83 .389 .75 50.19 17.35 .51 1.57 
Mixed relations 76.03 28.65 99.47 34.30 .381 .20 101.00 34.71 33 .21 
Color naming 53.81 8.99 62.09 8.17 .15+.09 62.07 9.80 .15 .09 
Hard directions 110.01 28.21 186.25 60.00 .69 1.13 188.72 53.91 .71 .91 
No. checking 151.52 28.27 157.03 34.72 .04 .23 176.03 33.52 .16 .19 
Form subs. 124.40 13.11 150.92 21.70 .21 .65 152.58 30.46 .23 1.32 


No. c. Errors 4.70 $88 8.20 7.62 .74 1.25 7.95 7.88 .68 1.82 


Form s. ” 48 .67 £22.68 4.48 4.48 3.85 3.17 3.22 5.46 3.81 
Hard d. ” 164 .45 4.838 2.48 1.69 4.51 4.00 2.96 1.44 5.58 
No. of subjects 45 38 65 


Av.—average score of the group in seconds and hun- 
dredths of a second, in the tests from opposites down through 
form substitution.* In the last three tests not the time but 
the average number of errors and hundredths of an error 
is given. 

X—percentage that the average of the group is above 


or below the average of group 1. A + quantity indicates 


that the average is below the average of group 1, or a better 
performance, and no sign before the figures indicates that 
the average is above the average of group 1, or a poorer 
performance. 

Z—percentage that the standard deviation of the group is 
above or below the standard deviation of group 1. 


*Since in group 1 the grades in the number checking and form sub- 
stitution tests are measures in terms of the amount accomplished (av- 
erage in number checking—117.31, average in form substitution—76.31) 
the figures, in the table under group 1, for these two tests, are roughly 
derived, for means of comparison in this table only, by dividing the 
time (constant for the group in each case) by the amount done and mul- 
tiplying by the number of groups of figures in the number checking test, 
and the number of figures in the form substitution test. 
mee no. of cases. 


Groups 2 and 3 showed themselves to be greatly inferior 
to group 1 in performing the tests. Groups 2 and 3 were made 
up of girls who were working long hours for very low wages 
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and who came from families in poor circumstances. Twenty- 
four per cent of the subjects of group 2 and twelve per cent 
of the subjects of group 3 came from homes where a foreign 
language was spoken. The average foreign-born perform- 
ance, in each test, in each group, was compared with the 
average performance of the rest of its group and was found 
to be higher by no more than six per cent in any one test. 
The inferiority in performing the tests in comparison with 
group 1 could, therefore, not be traced to difficulty in using the 
English language due to speaking a foreign language in the 
home. 


B. Treatment of errors. 

At the outset of the experiment it was planned to treat 
separately the time required to perform the test and the 
number of errors made and to correlate the number of errors 
made in the number checking, form substitution, and hard 
directions tests with the grades in stenography, grammar, 
and typewriting. The standard deviations of the number of 
errors made in each of the tests being in every case almost 
as high if not higher, than the average number of errors made, 
no attempt to correlate the errors with the work has been 
made since the number of errors made is so unreliable as a 
measure. 


C. Methods of combining the measures. 

When several tests have been made of an individual’s 
abilities it is often desirable to show the success of the in- 
dividual in the series of tests taken as a whole. A good way 
of doing this is by the order of merit method. The measures 
of the individuals are so arranged that it can be stated that 
an individual stood thirteenth from the top in one test, 
twenty-third in another test, eighteeth in still another test 
and that her average rank was eighteenth. This method 
was used in correlating performances in the tests with ability 
in stenography and grammar. 

Although the order of merit method is rather a rough 
method it has worked well with certain kinds of material 
in the past. But to transmute a series of quantitative 
measures into an order of merit series is to throw away 
a great deal of information contained in the series. Another 
method consists in taking the average of the group as 
zero and expressing the individual’s standing as a devia- 
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tion above or below this average. The measure of deviation,. 
usually the standard deviation, is taken as the unit and. 
all deviations are expressed as multiples of this unit. 
What this method does is to assign to each individual a 
quantitative position in the distribution of the group. This. 
method was used in getting reduced measures in the tests. 
which were correlated with ability in typewriting in groups 
2 and 3 and with grades in typewriting in group 1. 


D. Correlations. 
a. Stenography and grammar. 


I. Method used. 
The rank differences formula was used in correlating the 
tests with abilities in stenography and grammar, 


aj O2@ 
: n(n?~1) 
II. Correlations. 

The following table gives the correlations with the tests 


and abilities in stenography and grammar. 


Steno Grammar 
a a oS a es ha bie aus hacwiae.s 36 37 
Number checking .........cccccces .07 22 
EN es. Jasin seinem ee ae 04 38 
NN SRL a's os od adh enews 23 35 
NNN cls wasp ebb hae 19 my | 
SE  PUDGRISUENOE 5 i. ee cow oh case .40 16 
SO sg se ive a 'emy FES 46 54 
NE NN es nk can c € we A since e 31 43 
i st Cuie'g oN SNe bees os ‘ena: a .40 
I i ad sweat e medi ge 45 45 


b. Typewriting. 
I. Method used. 


In correlating the tests with abilities in typewriting, in 
the three groups, a derivation of the Pearson formula 
was used (52), ;-2Avy aft ;a and b being the reduced 
measures in the test and typewriting. The following table 


gives the correlations of the tests with abilities in type- 
writing. 
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‘ II. Correlations. 


Group 1. Group 2. Group 3.- 
Q 7) 
ie?) 
ie E 
o  b va | 4 $ gee 
we 2 2 8 ae $ PF &- 8 
, Sn ae a we 5 FE os =) 
2 oo eS ss > 2) ; 
ae ee 8s ) » +» + 
y Gee a a Se 
OZ2Ak A 4 dZ & HR KR GB F 
Verb-object 41 43 46 65 21 57 .46 28 82 84 833 2. 
No. check. 45 .47 .387 .3-.01 30 .46 .28 4 .41 34 38 
Color naming 00 .43 .55 .41 29.61 .40 89 §9 40. 66 ae 
Action-agent 42 .48 .29 .41 .00 .40 389. 18 .29 .26 29 .28 
Agent-action .29 .19 .40 .81-.02 .28 .30 .02 .20 .24 18 .26 
Form subs. Sh ae she jet ee see ee . O86: 28 Ait 17. .20 
Hard dir. 11 .14 .19 .13 84 82 14 -.20 -.20 -.08 16 19 
Mixed rel. -.09 .21 .17 .25 .04 .00 .138 -.07 -.12 -.09 03 -.04 
Opposites 17 .11 .O7 .15 .19 54 12 -.09 -.09 -.11 -.07 .06 
No. of subj. 42 40 40 40 29 27 40 38 38 38 65 65 


E. Discussion of correlations. 


a. Significant tests for typewriting and significant tests: 
for stenography and grammar. 

In comparing the correlations obtained between tests and. 
typewriting with the correlations between tests and stenog- 
raphy and grammar, if a line is drawn horizontally across 
the paper in the middle of the list of tests, in each of the’ 
preceding tables, it will be observed that, in most cases, the 
tests which correlated best with stenography and grammar 


and worst with typewriting lie below this line and the tests 


which correlate best with typewriting and worst with 
stenography and grammar lie above this line. The two tests 
which are not language tests, number checking and form sub- 
stitution, do not correlate well, except in one instance, with 
stenography and grammar, but one correlates very well and 
the other fairly well with typewriting. 


b. Typewriting, group 1. 

In considering the first four months’ correlations in type- 
writing one is met with the uniformity of the results for any 
one test during those four months. If a test correlates well 
with one month’s measures of abilities in typewriting, it can 
be predicted that it will correlate well with another month’s 
measures; if a test correlates poorly with one month’s 
measures it can be predicted that it will correlate poorly 
with another month’s measures. 
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The March correlations with the verb-object, number 
checking, agent-action and hard directions tests show no 
correspondence with the correlations for these tests for the 
first four months. This can be accounted for as follows: 
The March typewriting test was an especially difficult one 
which probably means that it called for reactions which the 
subjects had not formed the habits of making or that it called 
for the functioning of processes which did not exist or which 
were not well developed. Since the exercise was exceptional 
it could be expected that the results of the exercise would be 
exceptional and it is quite possible that the correlations would 
be exceptional and that the uniformity of correlation would 
be absent. It will also be observed that the number of sub- 
jects dropped from forty to twenty-nine this month. 

The April correlations show a good degree of correspond- 
ence with the correlations for the first four months except 
for the form substitution, hard directions, and opposites 
tests. Here again it will be observed that only twenty-seven 
subjects took the exercise. This reduction in the number of 
subjects taking the exercise does not mean entirely a process 
of the elimination of the poorer typists; some of the the 
poorer typists had dropped out of the course and some of the 
best typists were in the University offices working as typists. 
Inspection of the table will show that in the first three tests 
in the list, those which correlate highest with typewriting, 
and correlations are usually higher each month as the sub- 
jects acquire more speed in typing. 


c. Typewriting, groups 2 and 3. 


In the tests which correlate well in group 2 the correla- 
tions with the best week’s measure of ability are higher than 
the correlations with the five weeks’ measure of ability, and 
the correlations with the best day’s measure of ability are 
higher than either the correlations of the five weeks’ or the 
best week’s measure of ability. In the same tests in group 3 
the correlations with the best day’s measure of ability are 
higher than correlations with the fifth and sixth weeks’ meas- 
ure of ability. The best day measures the ability when a spurt 
is made and the best spurt over a long period of time is prob- 
ably the best index of a subject’s ultimate ability that can be 
found in practice of groups of this sort. Since the correla- 
tions are higher in these short intense periods it is concluded 
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that the tests tend to measure the physiological ability in 
typewriting. 


d. Typewriting, groups 1, 2 and 3. i 

The seventh column in the table, the average of the cor- 
relations for October, November, December and February 
was obtained as a means of comparison of group 1 with 
groups 2 and 3 rather than comparing any one month’s cor- 
relations of group 1 with the correlations of group 2 and 3. 
In groups 2 and 8 the correlations, for the first four tests in 
the list, in every case, are lower than those of group 1. With 
slight exceptions the tests which are correlated well in one 
group are correlated well in other groups and the tests which 
are correlated poorly in one group are correlated poorly in 
other groups. 


F. Partial correlations. 

The four tests which were correlated highest with type- 
writing ability were used in deriving a multiple correlation 
with the measures obtained in the typewriting exercise of 
February in group l. 

The calculation of the regression equation with five vari- 
ables is,—Measure in typewriting — .3721 (measure in verb- 
object) + .0109 (measure in color naming) + .0848 (measure 
in number checking) + .0852 (measure in action-agent). 
Or for practical purposes, after multiplying by 24.3, a conven- 
ient constant; Measure in typewriting — 9 (measure in 
verb-object) + .3 (measure in color naming) + 2. (measure 
in number checking) + 2 (measure in action-agent). After 
performing the arithmetic this gives a correlation of .61 with 
a probable error of .07. 


a 
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6. Applicability of the results. 


The correlation method has been used to measure the 
degree of association between performance in the tests and 
‘typing ability and it is now necessary to obtain the line of re- 
gression or the line of best fit in order to predict, for any in- 
dividual in the group, her most probable performance in typ- 
ing from her performance in the tests. 

The accompanying diagrams, appended hereto, show fre- 
quency distributions in groups 1, 2 and 3 of the reduced meas- 
ures in the combination of the four tests, as obtained by the 
weights assigned by the method of partial correlation, and the 
‘reduced measures in typewriting, the former being plotted on 
‘the x axis and the latter on the y axis. From the general 
‘Sweep of the scatter diagrams, it is evident that, as the meas- 
ures in typewriting increase or decrease the measures in the 
combination of the four tests increase or decrease. The 
‘straight line AB, giving the best fit to the data, is fitted to the 
‘points in the diagrams by means of the formula y=mx-+b 
‘which for group 1 becomes y — .68x + .00b.* 

The expression best fit is seldom defined. Its significance 
varies with the problem in hand and it generally means a 
_ fit which is convenient and which for the problem to be solved, 
gives satisfactory results. The principle upon which the 
values of m and b are determined is so to choose m and b as 
to make the mean square deviation of the observations from 
the resulting straight line a minimum. The pertinancy of 
‘this principle for the problem of prediction is plain since 
‘when observations are distributed according to the normal 
‘law, the Tables of the Probability Integral enable us to com- 

*y—mx-+bory—y=r x x — X where J, X and cy, ox are the means 
.and the standard deviations, respectively of the y’s and x’s. 

109.5 

y — .00 = .61-9¢- x — .00b. 

= .68x of .00b. 
‘pute the probability of a deviation equal to any multiple or 
‘submultiple of the root mean square deviation. Moreover, 
as in all problems of prediction it is desirable to have the root 
‘mean square deviation as small as possible it is obvious that 
a straight line which fits given data so as to make the mean 
‘square deviation of the points from a straight line a minimum 
‘is, for the problem of predicting one variable from a knowl- 
edge of the other, a good fit to the data. 
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We have found the best fitting straight line connecting 
-performances in the tests and ability in typewriting by means 


of the formula y=mx-++b. For any given value of x, repre- 


senting a performance in the tests, we can predict, by means of 
this formula, the most probable performance which will be 
achieved in typing. The problem is not completely solved by 
finding the equation connecting the two variables. We are 
not content to predict the most probable values of y, but we 
wish to know, in addition, the degree of accuracy of the pre- 
dictions. The scatter formula supplies an answer to this lat- 
ter question. The formula § (scatter) = oy /7- (30)* shows 
the accuracy with which, using the equation to the best fitting 
straight line, the magnitude of the one variable may be pre- 
dicted from a knowledge of the other. Substituting the val- 
ues, for group 1, in the formula gives S=84.t When the as- 
sociation between the two variables is perfect, or when r = + 
1., 8 = O and all of the points lie on the straight line, and by 
means of the equation to the best fitting straight line, we can 
predict exactly the value of y corresponding to a given value of 


x. But it is an exceedingly rare case when r = + 1, and when 


r lies between these two limiting values, we can still predict 
the results with a knowledge of the probabilities in favor 
of the prediction. When the association between the two 
variables is not perfect, r falls between the limiting values 
+ 1, and the equation to the best fitting straight line enables 
the computation of the most probable value of y corresponding 
‘to a given value of x and the value S shows the accuracy with 
‘which, using this equation to the best fitting straight line, 
the magnitude of the one variable may be predicted from a 
knowledge of the other or enables us to say within what limits 
any proportion of the measures are scattered about the 
straight line. The degree of association between performance 
in the tests and typing ability is measured by the correlation 
formula whose value is r — .61: the formula by means of 
~which typing ability can be predicted from performance in 
the tests is y—.68x-+.00b: and the error of the predictions 
is measured by means of the scatter formula the value of 
which in this case is S = 84. 

It is known from the Table of the Probability Integral 


ythat when the distribution of the points about the straight line 


' *For the derivation of this formula the reader is referred to Chapter 2. 
+S = oy V1-12 = 106.5  T-.612 = 84. 
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is normal, 99.7 per cent of the observations lie within a de- 
viation from the straight line equal to + 38S, 95 per cent lie 
between + 28, and 68 per cent lie between + S. Since S = 
84, 2S = 168. In the chart for group 1 the lines + 2S are 
drawn parallel to the line AB and 168 points above and esis 
it, respectively, on the y axis. 

The accuracy of the forecasts is measured by S = oyj= 
where oy is the standard deviation of a concrete series, and r 
is the correlation between the two series. S measures the 
accuracy of the predictions because it shows how the predic- 
tion formula enables us to reduce their variability. If there 
were no predicting formula the variability of the series that 
we wish to know would be cy but by the use of the formula the 
variability of the predictions is only cy ji-. The factor /1-1? 
measures the reduction in variability that is gained by means 
of the predicting formula. If, therefore, we wish to compare 
the accuracy of predictions of two different series, cy being 
constant, the measure of the relative accuracy is given by 
yi-r and the smaller the value of i, the greater the ac- 
curacy of the predictions. The same idea may be put in a 
different way by saying that the greater the value of r, the 
greater the accuracy of the predictions. 


Scatter Diagram for Group 1. r=.61l. y = .676x + .00b. ae 
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Scatter Diagram for. Group 3. r= .45.. y= .515x-+:.00b. 
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In group 1 all but five subjects received passing grades in 
typewriting which, in terms of the diagram, means that all 
subjects scoring above —150 on the y axis accomplished sat- 
isfactory work. We will take this point, for the group, as 
being the lowest point to which any one subject may reach 
and still do satisfactory work in typing, then it can be predict- 
ed with 95 per cent accuracy that all the subjects who score 
+25 in the combination of the four tests will score between 
—150 and +185 in typewriting or will accomplish satisfac- 
tory work in typewriting (intersection of the line —2S with 
+25 on the x axis = —150 on the y axis and the intersection 
of the line +2S with +-25 on the x axis = +185 on the y 
axis). Or for any specific individual such as one, for instance, 
who receives a score of +225 in the tests, we can predict that 
she will be able to accomplish a score of +150 in typewriting 
(intersection of the line of best fit with +225 on the x axis = 
+150 on the y axis) and we know with 95 per cent accuracy 
that her accomplishment in typewriting will not be lower 
than —12 nor higher than +304 (intersection of the line 
—2S with +225 on the x axis = —12 on the y axis and the in- 
tersection of the line +2S with +225 on the y axis = +304 on 
the y axis). For convenience the reduced measures have been 
used. After the arithmetic of balancing the scores in the 
tests has been computed the same thing may be accomplished 
with the actual scores in the tests and in typewriting. 

There appears to be a great deal of difference in the opin- 
ions of psychologists as to just how much of a coefficient of 
correlation, with its probable error, constitutes a significant 
correlation, some psychologists having reported figures lower 
than .25 as being significant coefficients of correlation. Some 
of this, confusion can be cleared up by means of this method, 
and the practical significance of coefficients of correlation may 
be determined. 

The problem of vocational guidance is to give an individual 


counsel as to what sort of an occupation or profession his 


potential capacities fit him for, so that he can be definitely 
informed that he will be successful in certain lines of endeavor 
and unsuccessful in certain other lines, or in more specific 
cases for any one occupation such as assembling, typewriting, 
teaching psychology, etc., whether the individual will be a 
success or a failure and to just what degree he will be a success 
or a failure in that specific occupation. Unless the chances 
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of an individual succeeding or not succeeding are very accu- 
rately determined it does the individual very little good to 
know that his chances for success are such and such a percent, 
as seventy-nine or eighty-two percent, etc. For the problem 
in hand, in the typewriting occupation, this sort of counsel 
given to an individual may result in her spending an amount 
of time ranging from a few months upward in training and 
then she may finally learn that she is one of those unfortunate 
ones outside of the seventy-nine percent and that she will 
never become a successful typist. It is evident that of ten in- 
dividuals examined if we can predict that the chances for 
success in each case is, say, approximately eighty-nine per- 
cent and they are all counselled to take up the study of type- 
writing and go through a period of training and at the end 
of the training period nine develop into successful typists and 
one becomes a failure at typing then as far as this one in- 
dividual is concerned she has been misguided and her time and 
expense of training has been wasted to say nothing of the 
effect that the failure may have on her temperament and 
character. 

Vocational guidance should mean accurate scientific pre- 
diction for each individual and if, in the future, applied psy- 
chology, in the field of vocational guidance, would escape the 
many abuses that have been heaped upon it in the past it will 
not attempt to establish a system of vocational guidance 
until it can be almost perfectly sure of each and every one of 
its predictions. To establish such a system of vocational 
guidance which is to make no grave errors it will probably 
be necessary to have a coefficient of correlation well above 
ninety with a very small probable error and with a small scat- 
ter value; just what coefficient of correlation, probable error, 
and scatter value will be necessary will probably be impossible 
to accurately determine until the actual work has been accom- 
plished and the scatter diagrams drawn and interpreted. 

From the results of this research, it is interesting to know 
that we can predict that individual A who scores 140 in the 
psychological tests will most probably attain an ultimate abil- 
ity in typing of forty words per minute, and in 99.7 percent of 
the cases she will attain an ultimate ability in typing varying 
between not more than thirty-one to forty-nine words per 
minute, and that individual B, who scores ninety-six in the 
psychological tests will probably attain an ultimate ability in 
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typing of thirty-two words per minute and in 99.7 percent of 
all such cases she will, attain an ultimate ability in typing 
varying not more than between twenty-three to forty-one 
words per minute. But it will readily be seen that these. pre- 
dictions are of very little practical value for the individual 
in that they afford her very. little definite information about 
just what ultimate ability she, individually, will attain... From 
our data. we can readily predict that in all probability A will 
become a better typist than B but still it is quite possible for 
A to develop into a poorer typist than. B. The results of this 
research show, then, that the correlations are not high enough 
and the scatter value is not small enough to warrant the es- 
tablishment of a system of vocational guidance whereby an in- 
dividual can be advised, with any sufficient degree of safety, 
to undertake or not to undertake to prepare herself to. become 
a typist. | 

The problem of vocational guidance, then, is supplied. by the 
individual who wants to, know of what his potential, capacities 
consist and what sorts of occupations he is fitted. to follow 
successfully and what sorts of occupations he is not. fitted. to 
follow successfully. The problem of vocational selection is of an 
other sort, here the need is supplied by the profession or the 
industrial concern and not directly by the individual. The 
problem of vocational selection is—given a certain number 
of applicants in excess of the number. required for a specific 
job, to select those who, on the whole, are best fitted to succeed. 
In practical life the employment manager has a number of 
vacancies open for a specific job and of a limited number of 
applicants for that job his problem is to select from the ap- 
plicants those who, on the whole, are most likely to succeed. 
If ten jobs in the typewriting department are open and twenty 
typists are induced to apply for the jobs the problem is to se- 
lect the ten who, on the whole, will do the best work. If this 
can be accomplished there will be an ideal state of affairs. 
However, this ideal has never been reached in any system of 
vocational selection. If we have something which works 
with some degree of success, with such a degree of success 
never before attained by any other method, which has been 
checked up scientifically, then we are justified from a prag- 
matic view point in using it until a better method appears. 
If, we can have a system of vocational selection which can pick 
from the twenty applicants (ten poor ones and ten good ones) 


>" 
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a majority of the good ones and weed out a majority of ‘the 


poor ones we have something which works and which is val- 
uable... If by, our method of selection of -ten typists from 
twenty applicants eight finally’ turn out to be very good typ- 
ists, one a typist of medium ability, and one a poor typist then 
we have erred to some extent but on the whole we have made 
a selection which is a good one... By pure chance we would 
probably have selected five good typists and five poor typists 
and practically all of the traditional. methods of selecting 
typists appear to do nothing more than this even if ‘they ae- 
complish this much. | 
It:is evident from the foregoing data that this method of 
empirical vocational tests gives'a far more reliable criterion 
of vocational selection in the field of typewriting than: has 
been obtained by any other method. If a commercial concern 
finds it necessary to employ a number of typists atest in typ- 
ing may be given, as so many concerns do, which may or may 
not. determine the actual ability in typing at the moment. 
This will,insure the concern that they are not hiring: people 
who, at the moment are wholly inefficient. However, if we:are 
looking forward to the ultimate ability which the typists will 
achieve it is not safe to hire a person to do typing on the basis 
of the performance of a sample of typing for the typing abil- 
ity of a person when she first enters the employ of a concern 
is often no index of the ability that she may develop.})» Along 
with the test in typing the team of tests may be administered 


and from the results the employer may predict with 95-per- 


cent accuracy or with 99.7 percent accuracy, or with any other 
degree of accuracy that is required, whether or not the appli- 
cants will develop into good typists and in addition within just 
what limits their capacities in typing will enable them to de- 
velop in this field, these predictions being made: in. definite 
quantitative terms such as the net number of words written 
per minute.* The prediction of ability by means of the tests 
will do injustice to some individuals but it will vindicate it- 


tIn group 3 there was a coefficient of correlation of .28 with a prob- 
able error of .10 between the first day’s output and the best day’s output, 
the latter measure being taken within a period of six weeks. 

*The supply of typists in New York City in the last few years has 
been sufficient to enable a high degree of selection to be made. During 
the time in which the experiment was carried on in the commercial con- 
cern, in 1917, applications were received from two hundred and eighty- 
two typists for ninety-six positions which were open during that period. 
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self by the percentage of best applicants, and the percentage 
of poorest applicants which it will be able to detect. 
Some concerns require an average of twenty-two words of 
connected discourse to be written per minute, others require 
thirty-five words, others forty-three words, and so on. Some 
concerns require twenty words of unconnected discourse to be 
written per minute, other concerns require ninety-five strokes 
of indexing work to be written per minute and so on. There 
are then various degrees of quantitative and qualitative typ- 
ing required by various concerns and a typist who is capaci- 
tated to satisfy the requirements of one concern may not be 
capacitated to satisfy the requirements of another concern. 
The tests have developed a definite scale of efficiency in typing, 
from lower to higher grades. Since each of the various con- 
cerns requires a different average ability in its typists, higher 
or lower than that required by other concerns, it will be neces- 
sary, before the tests can be adequately applied, first to deter- 
mine this required standard for the given concern. Reference 
can then be made to the scale and a standard time for per- 
formance in the tests can be fixed upon. Candidates may then 
be selected in order, by their grades, from the highest scores 
down, until the required number have been secured. The em- 
ployment manager may stop at the “minimum score” in the 
tests corresponding to the minimum standard of efficiency 
required for the job and then attempt to procure a new group 
of applicants in order to secure applicants who can pass the 
tests with better than minimum grades, or if the supply of 
possible applicants is limited, he will at least have clear in- 
’ formation, at the start, as to what he may expect of those 
whom he is forced to select from the group who obtain inade- 
quate scores in the tests. Even here he is able to make the 
best of a bad situation. 
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APPENDIX 


1. The supervisor as a judge of abilities. 

In correlating abilities in performing tests with other abil- 
ities it is often impossible to obtain objective measures, of in- 
dividuals in a group, in exact terms of output or of a definite 
quality and quantity of a particular capacity or ability, in 
which case measures of abilities are estimated by a teacher or 
supervisor. The following analysis of a particular super- 
visor’s estimate of abilities may contribute, in some small 
way, to the reliability which is to be attached to such meas- 
ures. 

The supervisor of the work of groups 2 and 3 was asked to 
arrange the subjects in each group in an order of merit series 
for typewriting ability. Emphasis was placed upon the neces- 
sity of the supervisor basing his judgments on typewriting 
ability only and to exclude all other factors. The supervisor 
Was in posession of a copy of all the records of the output. 


The following table gives the correlations of his judgment 


with the measures of abilities according to output. 


Group 2. ?. 
Daily average for five weeks -74 
Daily average for best week 61 
Best day’s performance 59 
Group 3. 
Daily average for 5th & 6th weeks 53 
Best day’s performance 42 


The correlations tend to show that the supervisor was a 
better judge of ability which manifested itself over a long 
rather than a short period of time. He was either a poor 
judge of ability which manifested itself in spurts rather than 
ability which manifested itself in steadiness from day to day, 
or he based his estimates on something else than pure ability, 
or both alternatives may be true. In conversation with the 
supervisor, he realized that there were other things which 
he had taken into consideration in making the judgments, 
although he tried to judge for actual typing ability only, such 
as steadiness and general reliability of output, attendance, 
conduct, ability of the subject to make herself an agreeable 
member of the group, etc. In other words he was incapable 
of judging actual typing ability and his estimates were based 
on the general desirability of the subject to the concern. 
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This is a problem which every investigator who obtains: 
measures of abilities by means of some one’s estimate of those: 
abilities, must confront. Tt is probably impossible: for the 
general run of supervisors‘to grade for'pure ability and to 
omit all other factors: ‘This may be a favorable point if 
tests: are’ sought: which: will ‘be indices of an’ niger vinarm oni 
__ value ona job.) | . 


as “Averages of the scores in the tests arranged according to 
age and education. 7 
. In the following tables group M is made up of all the sub- 
jects who were tested in the Extension Department of Colum- 
bia University and group N is made up of all the subjects who 
were tested in the industrial concern, The number of subjects. 
in each sub- division is small and the deviations from the aver- 
age are high, so that, a good amount, of caution. must be exer-. 
cised in any interpretation. 

~When all the evidence is evaluated no difference with age’ 

and performance i in the tests is. found, 
_ Since the comparison, in group M, in education, is. made: 
between college graduates, high school graduates, and .gram- 
mar school graduates, only those subjects who had reached 
their twenty-second year are used in the group. ' The tables 
show that there is a difference between education and per- 
formance.in some of the tests, but where there is a difference 
it does not appear to be strongly marked. 
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rabl9¥ z 
.5 , ; oo 2 
x vei eee Gee 
4 pa. > ay 3 oa 
b: £ a o@ 5 r= a 
g a ee ee ae 
< A oGFw diivpboxicnage (soiagods JahW Oy 
16 1 35.8 . 306 29.4. 446 51.0. 107.4 
17 3 386.383 32.60 42.20 37.07 55.738 109.47 
18 14 28.47. 31.388 . 83.54. 34.70. 55.04 118.10 
19 12 31.42 34.28 35.58 35.35, 54.03 117.67 
20 8 28.68 30.65. 29.380 .32.45 52.65 110.75 
21 8 29.78 28.50 28.65 31.50 56.15 98.45 
22-25 10 25.56 28.06 26:74 30.42 50.14 89.86 
26-30 9 25.69 33.53 380.18 34.42 56.18 112.04 
31-44 12 28.08 36.19 29.92 87.12 52.52 132.98 
Group. N 
16 18 39.78 39.32 43.86 , 51.52..62.20 176.20 
17 40 38.92 36.79 38.35.. 47.16,,.62.21 177.40 
18 28 43.00 37.62. 39.23 51.07. .58.13 196.92 
19 14 41.13 35.74 35.47. 39,79.,.63.93 195.87 
20 5 42.88 37.36 33.72,..47.24..65.04 183.72 
21 15 44.40 88.88. 41.51 -48,36,,.62.33 176.60 
Groups M & N combined 
16 14 39.50 38.70 42.838 51.03 61.40 171.28 
17 43 38.74 36.49 38.62 46.46 61.67 172.66 
18 42 $8.14 35.54 87.33 45.61 57.10 170.65 
19 26 36.65 385.07 35.51 37.77 59.86 159.75 
20 18 31.06 33.23 31.00 38.14 57.41 138.82 
21 23 39.31 35.27 37.038 42.49 60.19 149.42 
EDUCATION 
Education Group N 
Gram. sch. grad. 54 44.51 39.12 41.74 51.03 63.12 186.76 
4 yr. high sch. 9 37.40 32.88 385.07 438.87 62.09 190.31 
1 yr. high sch. 21 35.81 39.01 35.85 41.43 59.17 187.62 
2 yrs. high sch. 19 42.15 386.81 48.25 48.82 62.51 195.14 
8 yrs. high sch. 8 35.50 32.05 387.05 50.38 59.82 154.53: 
4 yrs. high sch. 4 37.70 87.75 87.45 50.20 64.15 158.60 
combining, 
Gram. sch grad. 54 44.51 39.12 41.74 51.03 638.12 186.76 
Y%-1 yr. high sch. 30 36.29 387.17 35.27 42.16 60.05 188.43 
2-4 yrs. high sch. 81 39.28 35.39 40.90 49.40 62.01 179.94 
Group M 
Gram. sch. grad. 8 $4.04 30.64 38.84 45.386 59.88 180.08 
High sch. grad. 14 26.23 31.30 27.54 838.41 50.938 107.64 


College grad. 11 2511 2647 25.82 30.87 52.44 95.93 
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3. Sample of an Army Trade Test. (12).* | 


Welder, Cutter 


Cutter, Oxy-acetylene Operator a 
Oral _ 
Q. What is it called when the edges of metal sheets are welded to- 
__ gether in different spots to hold them in place for welding? 
A. Tack (spot). Score 4. 
Q. What chemical is mixed with water to form acetylene gas? . 
A. Carbide. Score 4, 
Q. What metal do you use to braze brass and cast iron together? 
A. Brass (bronze). ' Score 4. 
Q. What will happen if oil gets on the oxygen regulator or hose? 
A. F Explode (blow up). Score 4. 
(2) Catch fire (burn up). Score 4, | 
e «£ * * * 


Q. What is put in the acetylene tank to prevent explosions? 
A. Acetone. Score 4, 


Rating the candidate 


2 OE NE vss ke eMac de dbase cdean. 
We: SEC Ey ee ee SS 
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OR EE > RP Sons fron, eee: 
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Oe es MORCUIND isc neds vaioasvreia © 
ME TE a aalt Ware wablp oneceg’ hb ceaseee Oe 
EN ila iatiack A \a wie OWna eaten era a 
There is no E— or E+ rating. 


*Vol. 2, p. 148-150. 
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